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This paper presents the idea of multimodal teaching and learning and discusses how this 
perspective can help better understand the learning of students. The discussion is based 
on data gathered in a qualitative study of a fifth- grade bilingual classroom where at-risk 
students were successful in mathematics. We report on one class episode and one student 
as a case study for understanding multimodal learning. Analyses focus on how students 
use various texts such as speech and calculator keystrokes as resources to create 
meaning. This work suggests that a broader perspective and use of modes can support 
learning and provide students, especially those at-risk, with greater access to 
mathematics. 

Research in mathematics education over the last several years has included 
communication such as speech and written texts as important factors in learning (e.g., 
Steinbring, Bartolini Bussi, and Sierpinska, 1998). However, there also has been a 
tendency to assume that learning is primarily a linguistic accomplishment. Such thinking 
has obscured the fact that there is actually a multiplicity of modes of communication by 
teachers and students, including such modes as gestures and actions (Nunez, 2000), and 
that all of these modes express meaning and contribute to learning. Ignoring these other 
modes of communication or not bringing them to the forefront of consideration in the 
learning process, can blind us to key elements of the meaning making process. Moreover 
it can cause educators to overlook the full range of various resources students draw on to 
create mathematical meanings. The purpose of this discussion is to consider the idea of 
multimodal teaching and learning and to present a particular classroom episode to 
demonstrate how students fluidly and simultaneously use multiple texts to create meaning 
of a geometric problem. 

MULTIMODAL TEACHING AND LEARNING 

Our discussion and analyses draw on the work of Kress and his colleagues (e.g., Kress, 
Jewitt, Ogborn, and Tsatsarelis, 2001). In this work, communication refers to all 
meaning-making systems, or “modes”; these are organized, regular, and socially specific 
means of representation (Jewitt, Kress, Ogborn, and Tsatsarelis, 2001). Learning can be 
seen as a “...process in which pupils are involved in actively ‘remaking’ the information 
and messages (or complexes of ‘signs’) which teachers communicate in the classroom. In 
this way learning... is the pupils’ ‘reshaping’ of meaning (signs) to create new meanings 
(signs)” (Jewitt et al. 2001, p.6). Learning, therefore, is rooted in a dynamic process of 
sign making, but one which is devised, organized, and used according to social needs and 
practice (Halliday, 1985). Inherent in this conception of learning is that “...meaning 
arises as a consequence of choice and that meaning is multiple (Jewitt et al. 2001, p.6). In 
other words, when we make meaning, we choose to speak, use gestures, make drawings, 
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move our bodies, or use whatever resource available to communicate or represent our 
meaning, and we use multiple modes simultaneously. 

In classrooms, teachers use a complex of signs or an ensemble of communicative actions 
to convey specific meanings. Students remake these signs to convey their own meanings 
or messages of what they have learned. The richer the complex of signs, the more 
resources students can select and use to make new meanings. These resources or signs are 
integrated with those that the student already has from other experiences and knowledge 
including those from outside of school. Therefore, the representational resources 
available play a key role in mediating the meanings a student produces (Jewitt et al. 
2001). Students’ text, be it spoken or written, can be considered as ‘signs’ or evidence of 
their thinking or meaning making or choices they make in an active and complex 
transformation. In essence, a multimodal perspective provides a conceptual resource for 
observing crucial relationships among situated, embodied, and connectionist aspects 
within emergent forms of cognition. 

We apply these ideas to one students’ writing, which was part of a lesson solving a 
geometric problem. We first present a context that includes the problem that had to be 
solved, another student’s oral presentation of a solution, and then the writing. We analyze 
the writing by examining the connections and selection of resources this student made to 
convey his meaning. Our intention is to understand the thinking process the text 
represents in order to better be able to understand multiple modes of communication. We 
conclude with the implications of a multimodal perspective for learning and for teaching. 

CONTEXT OF THE STUDY AND THE DATA 

The data were gathered as part of a qualitative study of a bilingual fifth-grade teacher’s 
classroom (Chval, 2001). This teacher had a reputation for implementing standards-based 
mathematics and for consistently having high achievement gains in mathematics for a 
majority of her students. Unlike most classrooms in this school and neighborhood, 
students regularly used TI-30 calculators. In addition, students developed skills in 
speaking, reading, and writing calculator keystrokes. They used keystrokes to analyze the 
strategies of others, and to create and articulate reasoning to justify their solutions. 



=□ 

□ 



Figure 1: Violetta’s calculator keystrokes for problem 1. 

Late in the school year, the students were given two problems one of which was: “A 
three-quarter circle has an area of 100 square centimeters. Calculate the perimeter of the 
three-quarter circle.” Students had already solved similar problems involving full circles; 
however, the three-quarter element was completely new. Students worked individually 
and collaboratively to solve this problem. All of them solved the first problem, but only 
one student, Violetta, solved the second problem. As a result, Violetta volunteered to 
present the solutions for both problems at the board to the class. Violetta drew a sketch of 
a three-quarter circle on the board and then wrote the calculator keystrokes for her 
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solution to the first problem (see Figure 1). As she wrote each keystroke, she explained 
the meaning behind it. 

The following is a transcription of Violetta’s presentation of the first problem: 

01: Violetta: We are going to find the perimeter of the three quarter-circle. The area of the... 

The area of the three quarter-circle are 100 square centimeters. Now, we 
are going to go backward from the area to the perimeter. One hundred 
divided by three equals the area of one quarter-circle. Multiply by four 
to get the area of the whole circle. Divide by pi to get the area of the 
square built on the radius. You take the square root to get the radius. 
And then you multiply by two to get the diameter. Then we store it. [A 
reference to the student’s calculator.] Then we multiply by pi to get the 
circumference of the circle. Then we divide it by four to get the quarter- 
circle. Then we multiply by three to get the curvy part of the three 
quarter-circle. Then we sum it, sum it to memory. [Another reference to 
the calculator.] So we can get the circumference, the perimeter, of the 
three quarter-circle. 

After Violetta’s presentation and the resulting class discussion, students were asked to 
write about Violetta’s explanation to the problem. Juan’s final draft follows. 
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Figure 3. 



A MULTIMODAL ANALYSIS OF JUAN’S WRITING 
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Juan’s writing and expression of his mathematical understanding began when the 
problem was assigned and he went to work to solve it. Violetta’s talk contributed to his 
understanding as well. However, dialogue or talk can be considered only one part of a 
broader range of modes that contributes to Juan’s learning. For our purposes, we 
concentrate on understanding the interrelationships specifically between four modes of 
semiotic mediation: Juan’s mathematical writing (Written Text), the calculator keystrokes 
(Mathematical Symbols), Violetta’s presentation to the class regarding the same problem 
(Written & Spoken Text), and the three-quarter circle drawing (Geometric Figure). 

We suggest that meanings are being constructed as he moves continuously from the 
written text to Violetta’s spoken text to the geometric figure to the calculator keystrokes 
in no predetermined order. In this fluid process, Juan uses a variety of signs to create his 
written text. Each mode shapes his meanings as he constructs his text, but likewise, the 
written text becomes a mode itself that in turn shapes new meanings for him. To write his 
text he has to construct meanings regarding how these three modes of signs (Violetta’s 
talk, the geometric figure, and the keystrokes) relate to each other to form a coherent 
idea. 

We partitioned Juan’s writing into three sections: finding the area of the whole circle 
(Table 1); finding the diameter of the circle (Table 2); and finding the perimeter of the 
three-quarter circle (Table 3). This organizes the data and identifies the various modes of 
communication that were available to Juan as he wrote his text and the kind of meanings 
he constructed in this multimodal process. Each table contains Violetta’s talk in regards 
to her presentation to the class, the keystrokes that both Violetta and Juan wrote down 
either on the board or in Juan’s writing, and the geometric images of the three-quarter 
circle that Juan and Violetta refer to in their explanations. 

PART 1: FINDING THE AREA OF THE WHOLE CIRCLE 

We begin by looking at Juan’s written text regarding finding the area of the whole circle. 
Juan took what Violetta said, “one hundred divided by three equals the area of one 
quarter-circle” and changed it to “Violetta took the area of a three quarter circle and 
by three to get the area of the quarter circle.” For Juan the 100 (what Violetta said) and 
the area of the three-quarter circle (what Juan writes) have the same meaning. Juan also 
included the keystroke symbol ^ — to help him express his mathematical thinking. His 
choice to use the “divide” keystroke suggests he equates the word “divides” with the 
keystroke. Altogether, this leads us to believe that, in fact, one can map the calculator 
keystrokes to Juan’s written text (see Table 1). For example, every time Juan uses the 
expression “to get” in his text, this relates to the keystroke symbol “=”. Reciprocally, one 
may assume that Juan gave the equal symbol the meaning “to get.” The use of “to get” 
also suggests a connection between the keystroke steps and the mathematical concept the 
steps represent. 

Although Juan only included the picture of the three-quarter circle, we wondered what 
role the images of the three-quarter circle, one-quarter circle, and the whole circle played 
in the kinds of meanings Juan constructed in this part of his discussion. Over the course 
of the school year, the students had worked so much with circles in various forms that 
they seem to be able to visualize the image of a one-quarter circle. In essence, the 
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students have a strong conceptual history for solving the current problem, and this helps 
them in the current context. Therefore when Violetta says, “multiply by 4 to get the area 
of whole circle,” she is actually guiding her peers, including Juan, in forming an image of 
four quarter-circles forming a whole circle. In Table 1, we see that only the first 
geometric figure (the three-quarter circle) was evident in Juan’s writing, but clearly the 
other two images of circles were evident in Juan’s thinking. 



Violetta’s Talk on her pn 
to the class 


u The area of the three quarter circle are 100 square centimeters 
are going to go backward from the area to the perimeter. Om 
divided by three equals the area of one quarter-circle. Multiply 
get the area of the whole circle. ” 


Keystrokes 




100+3 = X 4 




Geometric Figures 


to to 


Juan’s Writing 


I am going to explain how Violetta went from the area of the thn 
circle to the perimeter. Violetta took the area of a three quarter 
by three to get the area of the quarter circle. Then she multipi 
area of one quarter circle by 4 to get the area of the whole circle. 



Table 1. Finding the area of the whole circle. 

PART 2: FINDING THE DIAMETER OF THE CIRCLE 



After Juan found the area of the circle, he divided by pi. In an earlier lesson, the students 
were given a circle with a diameter of 10 centimeters inscribed in a square with a side 
length of 10 centimeters. The square was partitioned into four equal squares built along 
the radius of the circle. Through various classroom activities students discovered that 
four squares built on the radius overestimated the area of the circle and three squares 
underestimated the area. As a result, students thought about building a square on the 
radius and multiplying by pi to calculate the area of circles. Thus, dividing by pi for Juan 
meant geometrically that he was finding the area of the square built on the radius (see 
Table 2). Again the historical context plays a critical role in Juan’s understanding. 

Also in Table 2, notice that in Juan’s writing he embeds the keystroke symbols (e.g., 4, jt, 
SUM) inside his narrative in a way that fits grammatically and syntactically with his 
explanation. We also can see that Juan uses different tenses (e.g., “multiplies,” “divided,” 
and “would have to add”) throughout his narrative. Writing is the most complex mental 
function, and it is misleading to think of writing as simply symbolic representation of 
speech (Wells, 1999) although there is a relationship between the two modes of language. 
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Both speech and writing express the same underlying meanings; however, writing is 
inherently a problem-solving situation fostered by attempts to create a visual 
representation of the meanings communicated in speech (Wells, 1999). Writing requires 
extraordinary deliberate analytical action which we see as Juan balances his written 
words (“area of a square built on a radius”) with Violetta’s spoken words, the keystrokes, 
and the geometric image of a quarter circle, all of which represent the area of a square 
built on a radius. 



Violetta's Talk on her pn 
to the class 


u The area of the three quarter circle are 100 square centimeters 
are going to go backward from the area to the perimeter. Om 
divided by three equals the area of one quarter-circle. Multiply 
get the area of the whole circle. ” 


Keystrokes 


100+3 = X 4 




Geometric Figures 


From to to 

CP ^ • 


Juan’s Writing 


I am going to explain how Violetta went from the area of the thn 
circle to the perimeter. Violetta took the area of a three quarter 
by three to get the area of the quarter circle. Then she multipi 
area of one quarter circle by 4 to get the area of the whole circle. 



Table 2. Finding the diameter of the circle. 



PART 3: FINDING THE PERIMETER OF THE THREE-QUARTER CIRCLE 

Finally, Juan and Violetta took the circumference of the whole circle and divided by four 
to find the length of one-fourth the circumference, which Violetta calls the “curvy part.” 
Then both students multiplied by three to get the “curvy part” of the three-quarter circle. 
The “curvy part” is represented geometrically in Table 3. Juan and Violetta used four 
keystrokes related to the calculator’s memory system, namely “SUM,” “STO,” “EXC,” 
and “RCL.” The “STO” key was used by both students to save the value of the diameter 
of the circle. Juan added his curvy part “to the STO” and Violetta added hers “to 
memory.” In both cases, the students used the “SUM” key in their keystroke text to add 
the value of the diameter to the value of the “curvy part,” but recovered the value of the 
diameter in slightly different ways as their keystroke text demonstrates in Figures 1 and 
2. Juan used “RCL” and Violetta used the “EXC” key, both of which give the same 
results. This is very important because it is evidence that Juan and Violetta are not simply 
copying text. 
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Also each student used different geometric images for combining the curve of the circle 
and the perpendicular radii, and represented the process in different linguistic styles. Juan 
combined the “curvy part of the three-quarter circle... and the two straight parts” (radii). 
Violetta, on the other hand, combined “...the curvy part of the three-quarter circle. Then 
we sum it, sum it to memory.” The memory she is referring to is the value of the diameter 
of the circle that she stored earlier. Even though they expressed the three-quarter circle 
differently, their keystrokes were the same for both of them with the exception of the last 
keystrokes (RCL and EXC). Clearly both Juan and Violetta seem to understand that the 
diameter is two times the radius and can be shaped geometrically by perpendicular radii 
to connect the three-quarter circle. Juan, therefore, is able to move between different 
ways of representing the diameter to make sense of the problem at hand and additionally 
balance spoken text, keystrokes, geometric images, and his own text. 



Violetta’s Talk on her pn 
to the class 


Then we multiply by pi to get the circumference of the circle, 
divide it by four to get the quarter circle. Then we multiply by th 
the curvy part of the three quarter-circle. Then we sum it, sum it U 
[Another reference to the calculator.] So we can get the circumfe 
perimeter, of the three quarter-circle. 


Keystrokes 


r>< pr in* i 

| X 1 3 II = \SUM II RCL | 


Geometric Figures 


O CP 


Juan’s Writing 


After this, she multinlies by \33L to get the perimeter of a whole 

and divided by to get the curvy part of a quarter circle. Fit 

multiplies by 3 to get the curvy part of a three quarter circle and 
to the sto to get the two straight parts. 



Table 3. Finding the perimeter of the three-quarter circle. 

CONCLUSION 



Our purpose has been to draw attention to the idea of multimodal teaching and learning in 
mathematics. The case we discussed presents an example of how students use many 
modes of signs as resources for learning or meaning making. This suggests that educators 
need to utilize multiple modes of communication and encourage students to utilize them 
also. Unfortunately, there is a tendency to inadvertently limit students’ access to and use 
of multiple modes as evidenced by the infrequent use of calculators, drawings, 
manipulatives, enactment, or even student presentations. For second language learners. 
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this infrequent use of multiple modes of communication actually limits their means to 
express their thinking and denies them more than one way to access mathematics. 
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